SIMULTANEOUS AVOIDANCE OF GENERALIZED PATTERNS 



SERGEY KITAEV AND TOUFIK MANSOUR 



Abstract. In [BabStein Babson and Steingrimsson introduced generalized 
permutation patterns that allow the requirement tha t two adjacent letters in 
a pattern must be adjacent in the permutation. In [ Kitl| Kitaev considered 
simultaneous avoidance (multi-avoidance) of two or more 3-patterns with no 
internal dashes, that is, where the patterns correspond to contiguous subwords 
in a permutation. There either an explicit or a recursive formula was given for 
all but one case of simultaneous avoidance of more than two patterns. 

In this paper we find the exponential generating function for the remaining 
case. Also we consider permutations that avoid a pattern of the form x — yz 
or xy — z and begin with one of the patterns 12 ... k, k(k — 1) ... 1, 23 . . . fcl, 
(k — l)(fc — 2) . . . Ik or end with one of the patterns 12 ... k, k(k — 1) ... 1, 
Xk(k— 1) ... 2, fcl2 . . . (fc— 1). For each of these cases we find either the ordinary 
or exponential generating functions or a precise formula for the number of such 
permutations. Besides we g enera lize some of the obtained results as well as 
some of the results given in |Kit3[ : we consider permutations avoiding certain 
generalized 3-pattcrns and beginning (ending) with an arbitrary pattern having 
either the greatest or the least letter as its rightmost (leftmost) letter. 



1. Introduction and Background 

Permutation patterns: All permutations in this paper are written as words 
7r = aia 2 . . . a n , where the dj consist of all the integers 1,2,... , n. Let a G S n and 
r G Sk be two permutations. We say that a contains t if there exists a subsequence 
1 < i± < %2 < • • • < ik < ti such that (o^ , . . . , on k ) is order-isomorphic to t, that 
is, for all j and m, Tj < r m if and only if <Zj . < cij m ; in such a context r is usually 
called a pattern. We say that a avoids r, or is r-avoiding, if a does not contain r. 
The set of all T-avoiding permutations in S n is denoted by S u (t). For an arbitrary 
finite collection of patterns T, we say that a avoids T if a avoids each r G T; the 
corresponding subset of S n is denoted by S n (T). 

While the case of permutations avoiding a single pattern has attracted much 
attention, the case of multiple pattern avoidance remains less investigated. In 
particular, it is natural, as the next step, to consider permutations avoiding pairs 



of patterns r l5 t 2 . This problem was solved completely for t\, r 2 € S3 (see [schSim ), 
for n G S3 and r 2 G ^4 (see [ |wf), and for n , r 2 G 64 (sec H , [k| and references 



therein). Several recent papers |CW| , |MVl| , [Krj , |MV3| , |MV2| deal with the case 



t"i G 5*3, t 2 G Sk for various pairs ti, t 2 . 



Generalized permutation patterns: In BabStein] Babson and Steingrimsson 
introduced generalized permutation patterns (GPs) where two adjacent letters in a 
pattern may be required to be adjacent in the permutation. Such an adjacency re- 
quirement is indicated by the absence of a dash between the corresponding letters 
in the pattern. For example, the permutation n = 516423 has only one occurrence 
of the pattern 2-31, namely the subword 564, but the pattern 2-3-1 occurs also in 
the subwords 562 and 563. Note that a classical pattern should, in our notation, 
have dashes at the beginning and end. Since most of the patterns considered in 
this paper satisfy this, we suppress these dashes from the notation. Thus, a pattern 
with no dashes corresponds to a contiguous subword anywhere in a permutation. 
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The motivation for introducing these patterns was the study of Mahonian statistics. 
A number of results on GPs were obtained by Claesson, Kitaev and Mansour. See 
for exam ple |Claes| , jKitl| , [Kit2| , jKit3[ and |VIansl| , |Mans2| , |Mans3 |. 

As in [SchSim], dealing with the classical patterns, one can consider the case 
when permutations have to avoid two or more generalized patterns simultaneously. 
A complete solution for the number of permutations avoiding a pair of 3-patterns 
of type (1,2) or (2,1), that is, the patterns having one internal dash, is given 
in [ ClaesMans | . In [Kitl] Kitaev gives either an explicit or a recursive formula 
for all but one case of simultaneous avoidance of more than two patterns. This is 
the case of avoiding the GPs 123, 231 and 312 simultaneously. In Theorem |l| we find 
the exponential generating function (e.g.f.) for the number of such permutations. 

As it was discussed in [Kit3|, if a permutation begins (resp. ends) with the 
pattern p = pipz-.-Pk, that is, the k leftmost (resp. rightmost) letters of the 
permutation form the pattern p, then this is the same as avoidance of k\ — 1 patterns 
simultaneously. For example, beginning with the pattern 123 is equivalent to the 
simultaneous avoidance of the patterns (132], (213], (231], (312] and (321] in the 
Babson-Steingn'msson notation. Thus demanding that a permutation must begin 
or end with some pattern, in fact, we are talking about simultaneous avoidance of 
generalized patterns. The motivation for considering additional restrictions such 
as beginning or ending with some patterns is their connection to some classes of 
trees. An example of such a connection can be found in [Kit3, Theorem 5]. There 
it was shown that there is a bijection between n-permutations avoiding the pattern 
132 and beginning with the pattern 12 and increasing rooted trimmed trees with 
n + 1 nodes. We recall that a trimmed tree is a tree where no node has a single 
leaf as a child (every leaf has a sibling) and in an increasing rooted tree, nodes are 
numbered and the numbers increase as we move away from the root. The avoidance 
of a generalized 3-pattern p with no dashes and, at the same time, beginning or 
ending with an increasing or decreasing pattern was discussed in [Kit3|. Theorem^ 
generalizes some of these results to the case of beginning (resp. ending) with an 
arbitrary pattern avoiding p and having the greatest or least letter as the rightmost 
(resp. leftmost) letter. 

Propositions 4-15 (resp. 16 - 27) give a complete description for the number of 
permutations avoiding a pattern of the form x — yzoixy — z and beginning with one 
of the patterns 12 ... A: or k(k — 1) ... 1 (resp. 23 . . . kl or [k - l)(fc - 2) . . . Ik). For 
each of these cases we find either the ordinary or exponential generating functions 
or a precise formula for the number of such permutations. Theorem ^ generalizes 
some of these results. Besides, the results from Propositions 4-27 give a complete 
description for the number of permutations that avoid a pattern of the form x — yz 
or xy — z and end with one of the patterns 12 . . . k, k(k — 1) ... 1, Xk(k — 1) ... 2 
and kl2 . . . (k — 1). To get the last one of these we only need to apply the reverse 
operation discussed in the next section. The results of Theorems || and ^8] can 
also be used to get the case of ending with a pattern from the sets A™ m or A™ a2: 
introduced in the next section. 

Except for the empty permutation, every permutation ends and begins with the 
pattern p = I. To simplify the discussion we assume that the empty permutation 
also begin with the pattern 1. This does not course any harm since, to count 
the generating functions in question for this, we need only subtract 1 from the 
generating functions obtained in this paper. 



2. Preliminaries 



The reverse R(jr) of a permutation tt — a\ai . 
The complement C (n) is the permutation 61&2 . . 



. . a n is the permutation a n . . . aia\. 
b n where bi = n+1— a^. Also, RoC 
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is the composition of R and C. For example, #(13254) = 45231, C(13254) = 53412 
and R o C(13254) = 21435. We call these bijections of S n to itself trivial, and it is 
easy to see that for any pattern p the number A p (n) of permutations avoiding the 
pattern p is the same as for the patterns R(p), C(p) and RoC(p). For example, the 
number of permutations that avoid the pattern 132 is the same as the number of 
permutations that avoid the pattern 231. This property holds for sets of patterns 
as well. If we apply one of the trivial bijections to all patterns of a set G, then 
we get a set G' for which Aciji) is equal to Ac(n). For example, the number of 
permutations avoiding {123, 132} equals the number of those avoiding {321,312} 
because the second set is obtained from the first one by complementing each pattern. 

In this paper we denote the nth Catalan number by C n ; the generating function 
for these numbers by C(x); the nth Bell number by B n . 

Also, NP(n) denotes the number of permutations that avoid the pattern q and 
begin with the pattern p; G^(x) (resp. E^{x)) denotes the ordinary (resp. exponen- 
tial) generating function for the number of such permutations. Besides, r™ m (resp. 
pmaa;^ denQtgg the set of all fc-patterns with no dashes such that the least (resp. 
greatest) letter of a pattern is the rightmost letter; A™ m (resp. A™ aa: ) denotes the 
set of all fc-patterns with no dashes such that the least (resp. greatest) letter of a 
pattern is the leftmost letter. 

Recall the following properties of C(x): 



(i) C( X ) = - 1 - 



2x 1 - xC{x) 

3. Simultaneous avoidance of 123, 231 and 312 



The Entringer numbers E(n, k) (see (SloPlcJ, Sequence A000111/M1492]) are the 



number of permutations on 1, 2, . . . , n + 1, starting with k + 1, which, after initially 



falling, alternately fall then rise. The Entringer numbers (see [Ent]) are given by 

£(0,0) = 1, £(n,0) = 0, 

together with the recurrence relation 

E(n, k) = E(n, k + 1) + E{n -l,n- k). 

The numbers E{n) = E(n,n), are the secant and tangent numbers given by the 
generating function 

sec a; + tana:. 

The following theorem completes the consideration of multi-avoidance of more 



than two generalized 3-patterns with no dashes made in [Kitl 



Theorem 1. Let E{x) be the e.g.f. for the number of permutations that avoid 123, 
231 and 312 simultaneously. Then 

E(x) = 1 + a:(sec(:r) + tan(x)). 

Proof. Let s(n; i\, . . . , i rn ) denote the number of permutations n £ S n (123, 231, 312) 
such that 7Ti7T2 . • • 7r r „ = ii«2 ■ ■ - im and / : S n — > S n be a map defined by 

/(7T17T2 . . .7T„) = (tTI + 1)(tT 2 + 1) . . . (?T„ + 1), 

where the addition is modulo n. Using / one can sec that for all a = 1, 2, . . . , n — 1, 
(2) s(n;a) — s(n;a + 1). 

Thus, |5 n (123, 231, 312) | = ns(n; 1) and we only need to prove that s(n; 1) = i? n -i, 



where E n is the nth Euler number (see [BloPlo, Sequence A000111/M1492]) 
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Suppose 7r G SVi(123, 231, 312) is an n-permutation such that m = 1 and 7T2 = t. 
Since ir avoids 123, we get tt^ < t — 1 and it is easy to see that 

t-l t-2 

s(n; l,t) = J^s(n; l,i, j) =^s(n- l;i- 1, j), 



so 



t-l t-2 

s(n; 1, £ + 1) = s(n; + VVn- 1;*, j) - Yjs(n- - 1, j). 



t-2 



Using (@) we get 

s(n; 1, £+ 1) = s(n; l,*) + s(ra-l;t, 1) + s(n- 1; t- 1, j - 1) -V] s(n- 1; t- 1, j), 

J=2 j=l 

and by (|^) again, we have for all t = 2, 3, . . . , n — 1, 

s(n; l,i + 1) = s(n; 1, t) + s(n - 1; 1, n - t + 1). 
Besides, by the definition, it is easy to see that s(n; 1,2) =0 for all n > 3, hence 

n 

using the definition of Entringer numbers [Ent] we get s(n; 1) = s n -,i.t = £7 n _i, 

t=2 

as required. □ 



4. Avoiding a 3-pattern with no dashes and beginning with a pattern 
whose rightmost letter is the greatest or smallest 



The following theorem generalizes Theorems 7 and 8 in |Kit3|. Recall the defi- 
nition of EP in Section Q. 

Theorem 2. Suppose p 1} p 2 € T™ m and p x € Sfc(132), p 2 € ^(123). Thus, the 



complements C(p\),C(p2) G rj*! 
for k>2, 



id C(pi) e 5 fc (312) ) C(pa) e S fc (321). TTie 



^ 2 W = ^ 1) W = 



21 J.fc-1, 



J?* 



and 



^2 3 (^) = ^3 c 2 ( r ) w = 



(fc-l)!(l-/; e -* 2 /2^) 
e ./2 J£ e -t/2 f fc-l gin ( Vj^ + |)) ^ 

(fc- l)!cos(#r + §) 



Proof. To prove the theorem, it is enough to copy the proofs of Theorems 7 and 8 
in [Kit3|, since the fact that the first k— 1 letters of p are possibly not in decreasing 
order is immaterial for the proofs of that theorems. Thus we can get the formula 
for Ef 32 (x) and Ef 23 (x), and automatically, using properties of the complement, 
the formula for E 3 ^' ( x ) anc ^ ^32i\ x )i directly from these theo rems. However we 
give here a proof of the formula for Ef 32 (x) an( i refer to [ Kit3 , Theorem 8] for a 
proof of the formula for Ef 23 (x). 

If k = 1, we have no additional re striction s, that is, we are d ealin g only with the 
avoidance of 132 and, according to [ ElizNoy , Theorem 4.1] or [ Kit2| , Theorem 12], 



Also, according to |Kit3, Theorem 6], 

e\Ux) 



-x 2 /2 



1. 



l-£er*/*dt 

Let R n ,k (resp. F n ^) denote the number of n-permutations that avoid the 
pattern 132 and begin with a decreasing (resp. increasing) subword of length k > 1 
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and let tt be such a permutation of length n + 1. Suppose tt = air. If r = e then, 
obviously, there are i? ni fe ways to choose it. If |r| = 1, that is, 1 is in the second 
position from the right, then there are n ways to choose the rightmost letter in tt 
and we multiply this by Rk, n -ii which is the number of ways to choose a. If |r| > 1 
then r must begin with the pattern 12, otherwise the letter 1 and the two leftmost 
letters of r form the pattern 132, which is forbidden. So, in this case there are 



Ri,kF n -i_2 such permutations with the right properties, where i indicates 



E 

i>0 

the length of a. In the last formula, of course, Ri : k = if i < k. Finally we have 
to consider the situation when 1 is in the k-th position. In this case we can choose 
the letters of a in (^j) ways, write them in decreasing order and then choose r in 
F„_ fc+li2 ways. Thus 

(3) R n +l,k — Rn,k + nRn-l,k + ^ ( . ] Ri,kFn-i.2 + ( , \F n -k+l,2- 



i>0 x ' 

We observe that (^) is not valid for n = k — 1 and n = k. Indeed, if 1 is in 
the fc-th position in these cases, the term (J^^j F n -k+i,2, which counts the number 
of such permutations, is zero, whereas there is one "good" (n + l)-permutation in 
case n = k — 1 and n good (n + l)-permutations in case n = k. Multiplying both 
sides of the equality with x n /n\, summing over n and using the observation above 
(which gives the term a; fe ~ 1 /(fc— l)\ + kx k /k\ in the right-hand side of equality (|J)), 
we get 

d x k ~ x 



(4) = KaW + x + 1)^132^) + iE\Ux) +x + 1). ( (| . 

with the initial condition -Ef 32 (0) = 0. We solve this equation and get 

□ 

Remark 3. It is obvious that if in the previous theorem pi £ 5fc(132) and p2 
S fc (123), then E^ 2 (x) = E% 3 (x) = 0. 

5. Avoiding a pattern x-yz and beginning with an increasing or 

decreasing pattern 

In this section we consider avoidance of one of the patterns 1 — 23, 1 — 32, 
2 — 31, 2—13, 3 — 12 and 1 — 32 and beginning with a decreasing pattern. We 
get all the other cases, that is, avoidance of one of these patterns and beginning 
with an increasing pattern, by the complement operation. For instance, we have 

Proposition 4. We have 

^23 (*)=^32 (*) = { ee ^ . /fc=1 . 

Proof. We prove the statement for the pattern 1 — 23. All the arguments we give 
for this pattern are valid for the pattern 1 — 32. The only difference is that instead 
of decreasing order in r (see below), we have increasing order. 

Suppose k > 2. Let B n _k denote the number of n-permutations that avoid the 
pattern 1 — 23 and begin with a decreasing subword of length k. Suppose tt = air 
be one of such permutations of length n + 1. Obviously, the letters of r must be 
in decreasing order since otherwise we have an occurrence of 1 — 23 in tt starting 
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from the letter 1. If |cr| = i then we can choose the letters of a in ("J ways. Since 
the letters of r are in decreasing order, they do not affect a and thus there are Bi^ 
possibilities to choose a. Besides, if |oj = k — 1 and letters of a are in decreasing 
order, we get U™i) additional possibilities to choose 7r. Thus 



Multiplying both sides of the equality with x n /n\ and summing over n, we get 
the differential equation 



dx l ~ M y ' K l ~ M y ' {k-1) 



with the initial condition E^^^'" 1 (0) = 0. The solution to this equation is given 
by 

(5) B*^-\x) = (e*/(k- 1)!) [ X t k -'e-^ dt. 



If k = 1, then there is no additional restriction. According to [Claes, Prop. 2] 
(resp. [Claes, Prop. 5]), the number of n-permutations that avoid the pattern 1-23 
(resp. 1-32) is the nth Bell number and the e.g.f. for the Bell numbers is e eX_1 . 
However, all the arguments used for k > 2 remain the same for the case k = 1 
except for the fact that we do not count the empty permutation, which, of course, 
avoids 1-23. So, if k = 1, we need to add 1 to the right-hand side of (||): 

Ei- 23 (x) = e e * f X e- e ' +t dt+l = e e *-\ 
Jo 

□ 



Proposition 5. We have 

E^-\x) = 



nl 

n>k-l 



e e ifk=l. 



Proof. Suppose k > 2. Let B n ^ denote the number of n-permutations that avoid 
the pattern 3—12 and begin with a decreasing subword of length k. Suppose 
7r = a(n + l)r be such a permutation of length n + 1. Obviously, the letters of r 
must be in decreasing order since otherwise we have an occurrence of the pattern 
3—12 in 7r starting from the letter (n + 1). If \a\ = i then we can choose the letters 
of a in r?) ways. Since the letters of r arc in decreasing order, they do not affect a 
and thus there are Bi : k possibilities to choose a. Besides, if n > k — 1, then it can 
be decreasing, that is, (n + 1) can be in the leftmost position. Thus 



B n +i,k — E ( • )Bi,k + S n< k, 

where 

. J 1, if n > k — 1, 
Vfc ~ \ 0, else. 

Multiplying both sides of the equality with x n /n\ and summing over n, we get 
the differential equation 

n>fc-l 
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with the initial condition E^_ 12 "' 1 (0) = 0. The solution to this equation is given 
by 

(6) Eg£>"-\x) = S / e-' £ hdt. 

J ° n>k-l 



If k = 1, then there is no additional restriction. In [Claes, Prop. 5] it is shown 
that E\_ 32 (x) = e e . Using the complement, the number of n-permutations 
that avoid 1 — 32 is equal to the number of n-permutations that avoid 3 — 12. We 
get that E\_ 12 (x) — e e -1 . However, all the arguments used for the case k > 2 
remain the same for the case k — 1 except the fact that we do not count the empty 
permutation, which avoids 3-12. So, if k = 1, we need to add 1 to the right-hand 
side of (§): 



E\_ 12 {x) = e e * / e" e e* dt + 1 = e e 
Jo 



□ 



Proposition 6. We have 

f 0, ifk>i, 
El%- 1 1) - 1 (x) = \ e* fte-*\e*-l)dt, ifk = 2, 
{ ifk = l. 

Proof. For k > 3, the statement is obviously true. If k = 1, then the statement 



follows from [ClaeE, Prop. 2] and the fact that there are as many n-permutations 
avoiding the pattern 1 — 23, as n-permutations avoiding the pattern 3 — 21. For 
the case k = 2, we can use exactly the same arguments as those in the proof of 
Proposition]^ to get the same recurrence relation and thus the same formula, which, 
however, is valid only for k = 2. □ 

Recall the definition of N'P in Section ||. 

Proposition 7. We have 



v ' 10, else. 



Proof. If k = 1, then the statement follows from Claes, Prop. 22]. Suppose k > 2 
and let 7r = gut be an n-permutation avoiding 2 — 31 and beginning with the 
pattern k(k — 1) ... 1. Suppose, without loss of generality that a consists of the 
letters 1,2,... ,£. Now I must be the rightmost letter of a, since otherwise £, 
the rightmost letter of a and n form the pattern 2 — 13. Also, the letter (£ — 1) 
must be next to the rightmost letter of a since otherwise the letter [l— 1), next 
to the rightmost letter of a and the letter I form the pattern 2 — 13. And so on. 
Thus a must be increasing, which contradicts the fact that 7r must begin with a 
decreasing pattern of length greater than 1. So |oj = and r must begin with 
the pattern (k — l)(k — 2) ... 1. Now, we can consider the letter (n — 1) and, by 
the same reasoning, get that it must be in the second position of tt. Then we 
consider (n — 2), and so on up to the letter (n — k + 2). Finally, we get that 
7r = n(n — 1) . . . (n — k + 2)tt', where tt' must avoid the pattern 2—13 and thus, 



there are C n ~k+i ways to choose tt (| Claes, Prop. 22]). □ 



Recall that C(x) is the generating function for the Catalan numbers. Also recall 
the definition of G v q in Section |^. 
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Proposition 8. We have 

n k{k-i)...i, s_J x k C k+1 (x), ifk>2 
° 2 - 31 W- \ C(x), ifk = l. 

Proof. If k = 1, then there is no additio nal res triction, and thus G\_ 31 (x) = C{x 



(applying the complement operation to | Claes , Prop. 22]) 



Suppose k > 2. Using the reverse, we see that beginning with k(k — 1) ... 1 and 
avoiding 2 — 31 is equivalent to ending with 12 ... k and avoiding 13 — 2, which by 



Claes] is equivalent to ending with 12 . . . k and avoiding 1 — 3 — 2. 

Suppose 7r = ir'nit" ends with 12 ... k and avoids 1 — 3 — 2. Each letter of n' 
must be greater than any letter of it" , since otherwise we have an occurrence of 
the pattern 1 — 3 — 2 involving the letter n. Also, it 1 and 7r" avoid the pattern 
1 — 3 — 2, and 7r" ends with the pattern 12 ... k. In terms of generating functions 
(the generating function for the number of permutations ending with 12 ... k and 
avoiding 1 — 3 — 2 is, of course, G2-31 ^l 2 -)) this means that 

(7) G k 2 %-^-\x) = xC{x)G^- 1 1) ---\x)+xGt^- A {x)i 

where the rightmost term corresponds to the case when it" is empty. Now, (|l|) 
and (0) give 

G k 2-3i ) "' 1 (x) = x k C(x)/(l - xC(x)) k = x k C k+1 (x). 

□ 

6. Avoiding a pattern xy-z and beginning with an increasing or 

decreasing pattern 

First of all we state the following well-known binomial identity 

n—m—k-\-l / -\ / - -1 \ / 

1 M ^ ^fc-lj^ m ) = \m + k 

Let s q {n) denote the cardinality of the set S n (q) and s 9 (n; ii, 12, . . .i m ) denote 
the number of permutations tt € S n (q) with 7Ti7T2 . . . 7r m = . . . i m . 

In this section we consider avoidance of one of the patterns 12-3, 13-2 and 23-1 
and beginning with an increasing or decreasing pattern. We get all the other cases, 
which are avoidance of one of the patterns 32-1, 31-2 and 21-3 and beginning with 
an increasing or decreasing pattern, by the complement operation. For instance, 
we have iVffr^Cn) = N^^in). 

6.1. The pattern 12 — 3. We first consider beginning with the patternp = k . . . 21. 



In [ClaesMans, Lemma 9] it was proved that 



3=0 



\i:2-3(Vi;i) C . 1 )s 12 - 3 {n - 2 - j), 



J 



together with si 2 _3(^; n) = si 2 -z{n\ n — 1) = s i2 _3(n — 1). 
On the other hand, from the definitions, it is easy to see that 



n-fc+l 



i=l 



Hence, using (g) and the fact shown in [Clacsj that Si2-3(n) equals B n , we get the 
following proposition. 
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Proposition 9. For all n > k + 1, we have 



N 



fc(fc-i)...i 

12-3 



(n) = (k + l)B n - k + 



i-k-2 



+ xT (u,) - ^rn - (V)) 



5 



to— fe— 1— j ) 



together with N^-^^'^ik) 



1 and A^^g = /or all n < k — 1. 



Now, let us consider beginning with the pattern p = 12 ... fe. From the defini- 
tions, it is easy to see that Null's (n) — for all n, where k > 3, and Nl 2 _ 3 (n) — 
s i2-3( n ) = 5„ (sec [ClacsMans, Prop. 10]). Thus, we only need to consider the 
case k = 2. 

Suppose 7r S S'i2-3(^) is a permutation with 7Ti < tt2- It is easy to see that 
7T2 = n. Hence Nl$_ 3 (n) = (n — l)si2-3(n — 2), for all n > 2, and by ClaesMans , 
Prop. 10], we get the truth of the following 



Proposition 10. We have 



( 0, 



3=0 



E 

L d>0 



d>0 
rr,d 



(l-x)(l-2x)...(l-dx) 



{I - x){\ - 2x) . . . (I - dx) [ 



ifk>3, 
, ifk = 2, 
ifk = l. 



6.2. The pattern 13 — 2. Let us introduce an object that plays an important role 
in the proof of the main result in this case. For n > m + 1 > 0, we define 



A(n; m) 



E 



Sl-3-2(n;ii,«2, ■ ■ ■ ,*m)- 



We extend this definition to m = by A(n; 0) = Si_3_2(n) 
Lemma 11. For all n > m > , 

' m + 1 — j 



J 



si_ 3 -2(n - j). 



Proof. For m = the lemma holds by definitions. Let m > 0; so 

n 

A(n;m) = ^ E s i-3-2(n; ii, ii,---, i m ,j), 

= A(n;m + l) + ^ si_ 3 _ 2 (n; ii, i 2 , . . . , i m , n), 

l<i m <— <ia<*i<n-l 

= A(n;m + l) + ^ Si_ 3 _ 2 (n - 1; h, i 2 , ■ ■ ■ , i m ), 

l^im <---<i2<H< n— 1 

= A(n;m + l)+ ^ si_ 3 _ 2 (n - 1; n - 1, i 2 , • • • , «m)+ 

l<i m <-<»2<rs-2 

+ E si_ 3 _ 2 (n- l;h,%2, ...,i m ), 

l<i m <— <i 2 <ii<n-2 

= m + 1) + A(n - 1; m)+ 

+ E si_3_2(n - 2; ii, . . . ,i m -i), 

l<i m _i<---<ii<n— 2 

= ■ • ■ = vl(n; m + 1) + A(n — 1; m) H h A(n - m - 1; 0). 

Hence, using induction on m, we get 
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A(n i m + l) = £(-iyr ,+ j 1 ■'j»i-,,- 2 (n-i) 

■±KM m ~ i t 1 ~ i )«*-0'- 1 -<-n- 



j>Q 



d=0j>Q \ •> / 

Using the identity Q - ([) + ■■■ + (-l)'Q = (V), we get 

A(n; m+l) = (™ + 1 " J ) fli- 3 - 2 (n - j) 

j>0 \ 3 / 



E(-i)i m ;^i-3- 2 (n-i-d). 



d 

Now using the identity (™) + (^"J = ("+ 1 ), we get 



A(n- m + 1) = ( ™ + 2 J «i-3-a(« - i). 

j>0 

which means that the lemma holds for to + 1. □ 

Now we find ■Ar^^ 2 1 ^'" 1 (n). 
Proposition 12. Let fc > 1. For all n > 0, 



d=0 j>0 

Proof. Claesson [Claes proved that the set of permutations that avoid the pattern 
13 — 2 is the same as the set of permutations that avoid the pattern 1 — 3 — 2, hence 

(9) N^ 2 1) "\n) = N^\zt\n). 

If the leftmost letter of a permutation avoiding 13-2 and beginning with the 
pattern fc(fc — 1) ... 1 is n, then, obviously, there are iVj*g_,j 2 ^ " " * 1 (n — 1) such 
permutations. Otherwise, it is easy to see that there are fc) such permutations. 
So, by Lemma [ll] and the considerations above, also using the fact that the number 
of (1 — 3 — 2)-avoiding rt-permutations in S n is C n , we get 



y;:;^-->) = N^ k - 2) -Hn - 1) + £(-D J ( fc + 1 - ') a-, . 

n^n \ 3 / 



rfc(fc-l) 

V"7 — J, 13-2 V"- ~~ v t 

j>0 

Moreover, using the definitions and Equation (^|), we have Nl 3 _ 2 { n ) = s i-3-2(n) = 
C„, for all n > 0. Hence, by induction on fc, the proposition holds. □ 

Now, let us consider the case of AT^ijj f(n). 
Proposition 13. Let k > 1. For a// n > k, we have 

#13-2 (n) = C n+1 ^ k . 
Proof. Suppose tt = ir'nu:" is a permutation in iS„(13 — 2) = S n (l — 3 — 2) (see ®)), 
such that 7Ti < 7T2 < • • ■ < 7Tfe. It is easy to see that there exists an to such that 

7r = (to + l)(m + 2) . . . (m + fc - l)/3mr", 

where /3 is a 1 — 3 — 2-avoiding permutation on the letters m+k, m+fc+1, . . . , n— 1, 
and 7r" G 5 m (l — 3 — 2). Hence, in terms of generating functions, we get 

J2 N iti{n)x n =x k C 2 (x). 

n>0 

The rest is easy to check using the identity xC 2 (x) = C(x) — 1. □ 
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6.3. The pattern 23 — 1. We first consider beginning with the pattern p = fc(fc - 
!)...!. 



Proposition 14. For all k > 1, 

^fc(fe-l)...!. 



E. 



23-1 



\x) 



Jc-1 



v d>0 



(1 - x)(l -2x) ■■■{!- dx) 



Proof. Let tt € 5„(23 — 1) be a permutation such that tt\ < tti < ■ ■ ■ < TTk- Since tt 
avoids 23—1, we have ttj = j, for each j = 1, 2, . . . , fc — 1. Hence 7r = 12 . . . (fc— l)7r', 
where tt' is a non-empty 23 — 1-avoiding permutation in SVi+i-fc- The rest is easy 
to get by using [ClaesMans, Prop. 17]. □ 

Now let us consider beginning with the pattern p = 12 ... k. 

Proposition 15. Suppose k > 1. For all n > k + 1. 



1 



n-fc-2 
B n -k + /"] 



n - 1 



i 



n— fe— 1— j ) 



with N$zf{k) = 1. 

Proof. In ||ClacsMans . Lemma 16] proved that for all 2 < i < 

r i-2 

i 



E 

3=0 



S23-l{n 



together with s 2 3-i(n;n) = s 2 3-i(^;l) = S23-i(^- 1) = B n _i. 
On the other hand, by the definitions, it is easy to see that 

i-fe+i 

S23-i(« -k + 



n — i 
k - 1 



Hence, using (JsJ) and the fact that ]Claes | s 2 3_i(n) is given by B ni we get the 
desired result. □ 

7. Avoiding a pattern xy-z and beginning with the pattern 
(fc-l)(fc-2)...lfc or 23 ... Ail 

In this section we consider avoidance of one of the patterns 12 — 3, 13 — 2, 23—1, 
21 — 3, 31 — 2 and 13—2 and beginning with the pattern (k — l)(k — 2) . . . Ik. The 
case when a permutation begins with the pattern 23 . . . kl and avoids a pattern 
xy — z can be obtained then by the complement operation. 



7.1. Avoiding 12 — 3 and beginning with (k — l)(k — 2) 
Proposition 16. We have 

T {k-l)(k-2)...lk i 



Ik. 



Jv 12-3 



n- 1 
fc-1 



B n - 



Proof. Suppose 7r = n'mr" avoids the pattern 12 — 3 and begins with the pattern 
(k — l)(fc — 2) . . . lfc. We have that tv' must be decreasing, since otherwise we have 
an occurrence of the pattern 12 — 3 involving the letter n, and tt" must avoid 12 — 3. 
Also, since tt begins with (fc — 1) . . . 21fc, the length of tt' is fc — 1. Hence, by [Claes] 
(the number of permutations in S' n (12 — 3) is given by B n ) 1 we have 

1" 



N 



(fe-l)(fe-2)...lfc 
12-3 



(n) 



n 
fc-1 



B n -i... 



□ 
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7.2. Avoiding 13 — 2 and beginning with (k — l)(fc — 2) . . . Ik. By [Claes], a 
permutation tt avoids the pattern 13 — 2 if and only if tt avoids 1 — 3 — 2. 

Suppose tt = tt' mr" is an n-permutation avoiding 1 — 3 — 2 and beginning with 
(k — l)(fc — 2) . . . Ik. Obviously, tt' and tt" avoid 1 — 3 — 2 and each letter of tt' is 
greater than any letter of tt" , since otherwise we have an occurrence of the pattern 
1 — 3 — 2 involving the letter n. Also, tt' begins with the pattern (fc — l)(fc — 2) . . . lfc 
or tt' = (k- l )(fc - 2)... 1. 

By [ Knuth[ , the generating function for the number of permutations that avoid 
1 — 3 — 2 is C(x), hence, using the considerations above, 

G{ k-_iKk-2)...i k{x) = xG f-_f^)--^ {x)C{x) + x k C ( x ). 
Therefore, by (|l|), we get the following. 
Proposition 17. We have 

Hence 

, r (fc-i)(fc-2)...ifc, x _ / C n _(k_i), ifn>k 
^13-2 W -| Q) dse 

7.3. Avoiding 21 — 3 and beginning with (k — l)(fc — 2) . . . Ik. If k > 3 then, 
by the definitions, we have N^Z^ 1 ^ " lk {n) = 0. 

If k = 1 then, by the definitions and | Claes| , we have N^i-^in) = B n . 

Suppose k = 2 and tt = tt'titt" is an n-permutation avoiding the pattern 21 — 3 
and beginning with the pattern (k — l)(k — 2) . . . lfc = 12. It is easy to see that tt' 
must be increasing, and the length of tt' is at least 1. Thus, using the fact that the 
number of permutations in 5„(21 — 3) is given by B n (see | Claes| ), we have 

do) ^:s l)(fe - 2) - lfe (n)=£Y n : 1 V^ 



3=1 



J 



n-1 



3=0 



n-1 



Since B n = ( . )B n -i-j, equality (|10| ) gives that 



Ngzl )(k - 2) - lk (n) = B n -B n _ 1 . 



v 21-3 

Thus we have proved the following. 
Proposition 18. 

f 0, if k > 3 

N (k-i)(k-2)...ik {n) = I Bn _ Bn _ u ifk = 2; 

{ B n , ifk = l. 

7.4. Avoiding 23 — 1 and beginning with (k — l)(k — 2) . . . lfc. 
Proposition 19. We have 

ft + k - 3\ t=Z ft - 2\ 

t=2 ^ ' j=o \ j / 



. B„, ifk = l 



B n -u ifk = 2, 



Proof. Suppose fc = 2. We are interested in the permutations tt S 5„(23 — 1) that 
begin with the pattern 12. It is easy to see that tt\ — 1, hence B^ 1 ^) = -B n -i 
for all n > 2. 
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Suppose fc > 3. We recall that S23-i(n; t) is the nu mber of permutations in 
5„(23 — 1) having t as the first letter. By [ ClaesMans ] , s(n; 1) = £?„_i and for 
t > 2, we have 

t ~ 2 ft — 2 \ 



j'=o 



On the other hand, if a permutation tt = tt'Utt" avoids 23—1 and begins with 
the pattern (k — l)(k — 2) . . . Ik, then tt' is decreasing of length k — 2, and using 
s(n; t), we get 



A" 



23-1 



jfe-2 / " \ 7 



n — k— 1— j • 



□ 



i)(k 



lk. By [Clae; 



7.5. Avoiding 31 — 2 and beginning with (k 

permutation 7r avoids the pattern 31 — 2 if and only if tt avoids the pattern 3—1 — 2. 

Suppose 7r = 7r'l7r" is an n-permutation avoiding 3—1 — 2 and beginning with 
(k — l)(fc — 2) . . . Ik. Obviously, n 1 and 7r" avoid 3 — 1 — 2 and each letter of tt' is 
smaller than any letter of tt" , since otherwise we have an occurrence of the pattern 
3 — 1 — 2 involving the letter 1. Also, tt' begins with the pattern (k — l)(fc — 2) . . . lk 
or tt' = (k— l)(fc — 2) ... 2 and tt" is not empty. So, using the generating fu nction fo r 
the number of permutations avoiding the pattern 3 — 1 — 2, which is C(x) ([Knuthj), 
we get 



G 



(fc-l)0-2)...lfc, s _ r (k-l)(k- 
\ x ) — XLt 31-2 



.lfc 



31-2 



(x)C(x) +x k - 1 (C(x) - 1). 



Therefore, using (|l]), we get the following 
Proposition 20. We have 

(fc-l)(fc-2)...lfc 



G 



31-2 



"(X) 



x k C 3 (x), 
C(x), 



ifk>2, 
ifk=l. 



Hence 



N. 



-i)0- 

-2 



2). ..lfc 



(n) = 



Cn-k+2 
Cm 



c 



71 — fc+1 ■) 



ifk>2, 
ifk = l. 



7.6. Avoiding 32 — 1 and beginning with (fc — l)(fc — 2) . . . lfc. 
Proposition 21. 



N. 



(fc-l)(fc-2)...lfc 
32-1 



(n) 



0, 

B n _i-(n-2)B„_ 3 , 
B n - (n- l)S n _ 2 , 



i/fc >4 

if k = 3 and n > 3, 
if k = 2 and n>2, 

ifk = i. 



Proof. Using the definitions and [Claes , it is easy to see that the statement is true 
for k = 1, 2 and fc > 4. 

Suppose now that fc = 3 and tt — tt'Itt" is an n-permutation avoiding the pattern 
32 — 1 and beginning with the pattern (fc — l)(fc — 2) . . . lfc = 213. We have that 
tt' must be increasing, since otherwise we have an occurrence of the pattern 32—1 
involving the letter 1, and tt" must avoid 32— 1. Moreover, since tt begins with 213, 
the length of tt is 1 and the rightmost letter of tt" is greater than the letter of tt' . 
Also, it is easy to see that the number of permutations in 5„_i(32 — 1) beginning 
with the pattern 12 is the same as the number of permutations in S n (32 — 1) 
beginning with the pattern 213 (one can see it by placing 1 in the second position). 



Hence A^2_i (n) = £> n -i — (n — 2)B„_3 for all n > 3. 



□ 
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8. Avoiding a pattern x-yz and beginning with the pattern 
(fc-l)(fc-2)...lfc or 23... kl 

In this section we consider avoidance of one of the patterns 1 — 23, 1 — 32, 2 — 31, 
2—13, 3—12 and 1 — 32 and beginning with the pattern (k — l)(k — 2) . . . Ik. The 
case when a permutation begins with the pattern 23 . . . kl and avoids a pattern 
x — yz can be obtained by the complement operation. 



Proposition 22. We have 



E{ k_-m-2)...ik {x) 



1-32 K^J — \ n>fc-l 



ifk = l. 



Proof. Suppose k > 2. Let B n ^ denote the number of n-permutations that avoid 
the pattern 1—32 and begin with the pattern (fc— l)(k— 2) . . . Ik. Suppose tt = air is 
such a permutation of length n+ 1. Obviously, the letters of r must be in increasing 
order, since otherwise we have an occurrence of the pattern 1 — 32 in tt starting 
from the letter 1. If \a\ = i, then we can choose the letters of a in (™) ways. Since 
the letters of r are in increasing order, they do not affect a and thus there are Bi^ 
possibilities to choose a. Also, if n > k — 1, then 1 can be in the (A; — l)th position, 
and in this case, since tt begins with the pattern [k — l)(k — 2) . . . Ik, it must be 
that 7r = (k— l){k— 2) . . . 21fc(fc+ 1) . . . (n+ 1). Thus, in the last case we have only 
one permutation. This leads to the recurrence relation 



B n +i,k — E ( • )Bi,k + S n 
where 

„ J 1, if n > k — 1, 
dn < k ~\0, else. 

This recurrence relation is identical to the one given in the proof of Proposition^, 
so using this proof we get the desired result. □ 



Proposition 23. We have 



p (fc-l)(fc-2)...U 7 , _ j ££ Jo Jo (I=2)T er G drdt ' l f k - 2 > 

{X) -\e°*-\ ifk=l. 

Proof. If k = 1 , then the statement is true due to Proposition ^. 

Suppose k > 2. Let B n: k denote the number of rt-permutations that avoid the 
pattern 1 — 23 and begin with the pattern (fc — l)(fc — 2) . . . Xk. Suppose tt = air is 
such a permutation of length n + 1. Obviously, the letters of r must be in decreasing 
order since otherwise we have an occurrence of the pattern 1 — 23 in tt starting from 
the letter 1. If \a\ = i, then we can choose the letters of a in (™) ways. Since the 
letters of r are in the decreasing order, they do not affect a and thus there are 
Bi^k possibilities to choose a. Besides, if n > k — 1, then 1 can be in the (fc — l)th 
position, and in this case, since tt begins with the pattern (fc — l)(fc — 2) . . . lfc and 
r is decreasing, it must be that the fcth letter of 7r is (n + 1) and there are (^Zl) 
ways to choose the letters of a and then write them in decreasing order. Thus, 



B n +i,k — > ( . )Bi^k 



n\ „ In — 1 



^>\ij ' \fc- 2 

j>0 
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Multiplying both sides of the equality with x n /n\ and summing over n, we get 
the differential equation 

with the initial condition E[ k J 2 ^ k ~ 2 " > " lfc (0) = 0. If F(x) denotes the last term, 
then it is easy to see that F'(x) = y._ 2 y_ e x , an d thus 

px j.k-2 

FW = 1 W^y'' * 

Now, the solution to the equation above is given by 
(11) E^ {k - 2) --- lk {x) = e e * [ X e- £t F(t) dt = e e * f f f'^ e^ drdt. 



o Jo 



o (*-2)l 



For example, if k = 2, then (k — l)(fc — 2) . . . lfc = 12 and ([11]) gives 



Jo 

which is a particular case of Proposition ^, since the number of n-permutations 
that avoid the pattern 3—21 and begin with the pattern 21 is equal to the number 
of n-permutations that avoid the pattern 1 — 23 and begin with the pattern 12 by 
applying the complement. □ 

Proposition 24. We have 

0, if k > 3 

G (k-l){k-2)...lk( x) = ^ x 2 C 3 {x ^ lfk = 2 



Hence 



C{x), ifk=l. 



0, if k > 3 

JV (fc-i)(*-2)...ifc (n) = ^ Cn ^-C n ^ tfk = 2 



C n , if k — 1 . 

Proof. For the case k = 1, see Proposition 0. If fc > 3, then the statement is true, 
since in this case the pattern (k — l)(k — 2) . . . Ik does not avoid 2 — 13. 

Suppose now that k = 2. Using the reverse, we see that beginning with the 
pattern 12 and avoiding 2 — 13 is equivalent to ending with the pattern 21 and 



avoiding 31 — 2, which by [Claes is equivalent to ending with the pattern 21 and 
avoiding the pattern 3 — 1 — 2. 

Let 7r = 7r'l7r" be an n-permutation avoiding 3 — 1 — 2 and ending with the 
pattern 21. Obviously, it' and ir" avoid 3 — 1 — 2 and each letter of ir' is less than 
any letter of ir" , since otherwise we have an occurrence of 3 — 1 — 2 involving the 
letter 1. Also, it" ends with the pattern 21 or \ir"\ — 1. So, using the generating 



function for the number of permutations avoiding 3—1 — 2, which is C{x) ([Knuthj), 
we have 

G| 2 _ 13 (z) = xGl 2 _ 13 (x)C(x) + x(C(x) - 1). 
Therefore, using (Q), we get the desired result. □ 

Proposition 25. We have 

G < £s£ {k - 2) - X \x)=x k C 2 {x). 

Hence 

^)(*-^* (n) = /^-(*-x). ^ k 
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Proof. Using the reverse, we see that beginning with the pattern (k — l)(k — 2) . . . lfc 
and avoiding the pattern 2—31 is equivalent to ending with the pattern fcl2 . . . (k—1) 



and avoiding the pattern 13— 2, which, by [Claesj, is equivalent to ending with the 
pattern fcl2 . . . (k — 1) and avoiding the pattern 1 — 3 — 2. 

Let tt = tt'titt" be an n-permutation avoiding the pattern 1 — 3 — 2 and ending 
with the pattern fcl2 . . . (k — 1). Obviously, tt' and tt" avoid the pattern 1 — 3 — 2 
and each letter of tt' is greater than any letter of tt" , since otherwise we have an 
occurrence of the pattern 1 — 3 — 2 involving the letter n. Also, tt" ends with the 
pattern kl2...(k- 1) or tt" = 12 . . . (fc - 1). 

Using the reverse operation, the generating function for the number of permu- 
tations ending with the pattern fcl2 . . . (fc — 1) and avoiding 1 — 3 — 2 is equal to 

ms of generating functions, the considerations above lead 

to 

G (fc-i) ( *-2)...i* (a;) = xG i^m-^ {x)C{x) + x k C { x ). 

Therefore, by (|l|), we get the desired result. □ 
Proposition 26. We have 

#3-12 0) = \ 

I e e - 1 , ifk = l. 

Proof. Suppose k > 2. Let B n ,k denote the number of n-permutations that avoid 
the pattern 3—12 and begin with a decreasing subword of length fc. Let tt = 
a(n + 1)t be such a permutation of length n + 1. Obviously, the letters of r must 
be in decreasing order since otherwise we have an occurrence of 3 — 12 in tt starting 
from the letter (n+1). If |er| = i then we can choose the letters of a in (™) ways. 
Since the letters of r are in decreasing order, they do not affect a and thus there 
are -B^fc possibilities to choose a. Also, if \a\ = fc — 1 and the letters of a are in 
decreasing order, we get (^."jj additional ways to choose tt. Thus 

'n\ „ In 



jw = E( i) Bi - k+ [k-i 



i>0 

This recurrence relation is identical to the one given in the proof of Proposition ^, 
and we get the desired result using that proof. □ 



Proposition 27. 



ifk > 4 



E (fc-i)(fc- 3 )...ifc (n) = I ie e* /{k _ 1)l) f* t k-i e - e * + t dtj lfk = 2 ork = 3, 

{ e e "-\ ifk = l. 

Proof. If fc > 4 then the statement is true, since in this case the pattern (fc — l)(fc — 
2) . . . lfc does not avoid the pattern 3 — 21. In the other cases, we use the same 
arguments as we have in the proof of Proposition The only difference is that 
instead of decreasing order in r, we have increasing order. □ 

9. Conclusions 

The goal of our paper is to give a complete description for the numbers of per- 
mutations avoiding a pattern of the form x — yzovxy — z and either beginning with 
one of the patterns 12 . . . fc, fc(fc — 1) ... 1, 23 . . . fcl, (fc — l)(fc — 2) . . . lfc, or ending 
with one of the patterns 12 . . . fc, k(k - 1) . . . 1, lfc(fc - 1) . . . 2, fcl2 . . . (fc - 1). This 
description is given in Sections 5-8. However, some of our results can be generalized 
to beginning with a pattern belonging to r™ m or T™ ax , and thus to the ending with 
a pattern belonging to A™ m or A™ ax (see Section || for definitions). An example 



SIMULTANEOUS AVOIDANCE OF GENERALIZED PATTERNS 



17 



of such a generalisation is given in Theorem |28| below. This theorem generalizes 
Propositions |i| and ^6| and can be proved by using the same considerations as we 
do in the proofs of these propositions. 

Theorem 28. Suppose p 1} p 2 £ Tf in and Pl £ S k {l - 23), p 2 G S k (l - 32). Thus, 
the complements C(pi), C(p 2 ) G T™ ax and C( P i) £ S k (l - 23), C(p 2 ) £ S k (3- 12). 
Then, we have 

f (e e 7(fc-l)!) J^t k - 1 e- et+t dt, ifk>2, 
\ e e *-\ ' ifk = l. 
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